Speculative Execution for Information Agents

Practical deployments of information agents can suffer from sub-optimal performance and scalability for a number of reasons. In the case of web-based information integration, for example, data sources are remote and their latency can have a substantial effect on overall execution performance. Scalability can also be poor, since concurrent queries can cause multiple, simultaneous remote data retrievals (often of the same information), quickly consuming available bandwidth. The frequency of remote retrievals also makes such agents inherently I/O-bound, wasting CPU cycles. One way of optimizing execution in such scenarios is to engage in speculative execution. Tasks likely to be executed in the future can be performed in advance, such as when an agent is I/O-bound. Correctly guessing can be profitable – the overall end-to-end application could perform faster, bandwidth could be conserved, and the CPU could be scheduled more optimally. Still, designing a technique for speculative information agent execution is not simple. For one, there are competing justifications. For example, an agent could speculate based on the profile of a current user or on the activity of past clients, or on resource availability. Secondly, speculation itself incurs additional overhead: it should not interfere with normal execution and there must be some method of coordinating speculative execution between multiple agents. Third, speculative execution itself needs to be scalable hinting done on a per-user basis could lead to a prohibitive number of hints in popular agents. Consider an “information portal” application, where users can view news headlines and stock quotes and charts. Suppose that 90% of the users currently logged into the system have portfolios that contain Cisco Systems stock. In this case, we could speculate that at least one will request detailed information about this stock and thus we could pre-fetch this data, or just the costly part (for example, the chart graphic). Note that the idea of speculative execution is not limited to pre-fetching data. Other costly operators